Self-contained power and cooling domains

ABSTRACT

A method for providing for conditioning of a computer data center includes supplying a working fluid from a common fluid plane to a plurality of power/cooling units distributed across a data center facility in proximity to electronic equipment that is distributed across the data center facility; converting the working fluid into electric power and cooling capacity at each of the plurality of power/cooling units; and supplying the electric power to a common electric power plane serving a plurality of racks of the electronic equipment in the data center facility and being served by a plurality of the power/cooling units in the data center facility, wherein the common fluid plane serves at least 10 percent of the power/cooling units in the data center facility and the common electric power plane serves at most 5 percent of the electronic equipment in the data center facility.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/336,691, entitled “SELF-CONTAINED POWER AND COOLING DOMAINS,” filedJul. 21, 2014, which is incorporated herein by reference in itsentirety.

TECHNICAL FIELD

This document relates to systems and methods for providing power andcooling for areas containing electronic equipment, such as computerserver racks and related equipment in computer data centers.

BACKGROUND

Economical and flexible provision of electric power and cooling is animportant aspect of modern large-scale computer data centers. Such datacenters may house thousands, hundreds of thousands, or even millions ofcomputer servers mounted in numerous rows of computer racks, and thosecomputers may be used for a variety of computing functions. Thecomputers use electric power to perform the computations and otherrelated activities, and the power usage is relatively dense (megawattsof power in a single facility). As a result, the computer servers andrelated components generate a large amount of heat. For example, a pairof microprocessors mounted on a single motherboard can draw 200-400watts or more of power. Multiply that figure by several thousand (ortens of thousands) to account for the many computers in a large datacenter, and one can readily appreciate the amount of heat that can begenerated.

While the cost of generating the heat (in terms of electrical usage) canbe high for a data center, the cost of removing all of the heat can alsobe high. That cost typically involves the use of even more energy, inthe form of electricity and natural gas, to operate chillers,condensers, pumps, fans, cooling towers, and other related components.Heat removal can also be important because, although microprocessors maynot be as sensitive to heat as are people, increases in heat generallycan cause great increases in microprocessor errors and failures. In sum,such a system may require electricity to run the chips, and moreelectricity to cool the chips.

SUMMARY

This disclosure describes systems and methods that may be employed toprovide electric power within a data center facility and to remove heatthat is created at least in part through the use of that power bycomputers and associated equipment. In certain implementations,combination power/cooling units may be distributed relatively evenlythroughout a data center and close to the computer systems that need tobe powered and cooled. Each power/cooling unit may receive energy from aworking fluid such as steam, and may use that working fluid to generateelectricity. The output of such a generator system, other than theelectricity, may then be used to operate a cooling unit. For example,the steam may be converted into very hot water and can be used tooperate an absorption chiller or other cooling unit.

As noted, the power/cooling units may be widely dispersed within a datacenter, with, for example, a unit for each row of computer racks or morethan one unit per row of racks. For example, the units may be mountedabove a row of racks—either from the ceiling structure of the datacenter or mounted to the top of each rack. The units may also be mountedin certain racks themselves and may be positioned between racks ofcomputers to provide electricity and cooling to the adjacent computerracks. In yet other examples, the units may be mounted below an elevatedfloor and provide electricity and cooling to the racks of computersabove them. The relationship between power/cooling units and racks ofcomputers may be 1-to-1, 1-to-n, or n-to-1. Waste heat from the unitsmay be ducted into a plenum and then removed from a facility, such as byproviding stacks running upward from each unit into an enclosed atticspace or down into an underfloor warm-air plenum, either of which may beexhausted outdoors.

The various distributed units may also receive their working fluid froma single large fluid plane or domain. A plane or domain, in thiscontext, is an area through which an energy source (e.g., steam, naturalgas, or electricity) may flow naturally and readily, without beingblocked by structures such as valves, switches, transformers, and othercomponents. The benefits of having a relatively large common planeinclude an ability of energy to move readily from areas where it is notdemanded to areas where it is. Thus, for example, by providingdistributed power units like those discussed here throughout a datacenter facility, and tying the inputs of those units into a large singleplane, working fluid can move readily to units that are serving racksthat have a high current demand. Also, higher variability in demand maybe accommodated with less energy-generating capability, and overallutilization of energy generation in a facility may be higher.

The size of the electric power domains may be similarly large orsubstantially smaller than the working fluid domain. Where the electricpower domain is large (e.g., of the same size as the working fluiddomain, such as a majority of a data center facility), the outputs ofmultiple distributed power units may be tied together so thatelectricity can flow freely within and through the larger domain. Wherethe electric power domains are relatively small in size (e.g., servingonly a single-digit number of racks) and large in number, the failuredomains on the electrical side may be kept equally small. Thus, if afailure occurs in the electrical distribution for a particular domain,it will be automatically isolated and not affect electrical energydelivery to equipment served from other of the domains. In certainimplementations, automatic switches may be provided within a particulardomain and normally left closed, and then if a failure occurs in a partof the domain, the location of the failure may be identified, and theswitches may be opened under control of a central control system so asto isolate the failure into a small part of the system so that operationof equipment that was previously in the same domain as the failure mayagain be operated before the failure can be diagnosed and fixed.

In an example implementation, a method for providing for conditioning ofa computer data center includes supplying a working fluid from a commonfluid plane to a plurality of power/cooling units distributed across adata center facility in proximity to electronic equipment that isdistributed across the data center facility; converting the workingfluid into electric power and cooling capacity at each of the pluralityof power/cooling units; and supplying the electric power to a commonelectric power plane serving a plurality of racks of the electronicequipment in the data center facility and being served by a plurality ofthe power/cooling units in the data center facility. The common fluidplane serves at least 10 percent of the power/cooling units in the datacenter facility and the common electric power plane serves at most 5percent of the electronic equipment in the data center facility.

In a first aspect combinable with the general implementation, the commonfluid plane serves at least 50 percent of the power/cooling units in thedata center facility and the common electric power plane serves at most5 percent of the electronic equipment in the data center facility, andwherein the data center facility is rated at 10 MW or more.

In a second aspect combinable with any of the previous aspects, thecommon fluid plane serves at least 90 percent of the power/cooling unitsin the data center facility and the common electric power plane servesat most 10 percent of the electronic equipment in the data centerfacility, and wherein the data center facility is rated at more than 10MW.

In a third aspect combinable with any of the previous aspects, thepower/cooling units include an electric generator unit generating powerfrom the working fluid and supplying waste from the working fluid to acooling unit that supplies cooling fluid to cool the electronicequipment.

In a fourth aspect combinable with any of the previous aspects, theworking fluid includes steam and the waste includes hot water producedfrom the steam.

In a fifth aspect combinable with any of the previous aspects, theelectric generator includes a steam turbine.

In a sixth aspect combinable with any of the previous aspects, thecooling unit includes an absorption chiller.

In a seventh aspect combinable with any of the previous aspects, theelectric generator unit and the cooling unit are located in a unitaryhousing with each other.

In an eighth aspect combinable with any of the previous aspects, theunitary housing is located within a row of racks that include the one ormore racks of computers.

In a ninth aspect combinable with any of the previous aspects, theunitary housing is located above a row of racks that includes the one ormore racks of computers.

A tenth aspect combinable with any of the previous aspects furtherincludes exhausting heat from the power/cooling units to an areaexternal to the data center.

In another general implementation, a system includes a plurality ofpower/cooling units distributed across a data center facility inproximity to electronic equipment that is distributed across the datacenter facility; a common working fluid plane defined by a plurality ofinterconnected conduits arranged to serve a plurality of thepower/cooling units; and a common electric energy plane defined by aplurality of interconnected electric conductors and served by aplurality of the power/cooling units. The common working fluid planeserves at least 10 percent of the power/cooling units in the data centerfacility and the common electric power plane serves at least 10 percentof the electronic equipment in the data center facility.

In a first aspect combinable with the general implementation, the commonfluid plane serves at least 50 percent of the power/cooling units in thedata center facility and the common electric power plane serves at least50 percent of the electronic equipment in the data center facility, andwherein the data center facility is rated at more than 10 MW.

In a second aspect combinable with any of the previous aspects, thecommon fluid plane serves at least 90 percent of the power/cooling unitsin the data center facility and the common electric power plane servesat least 90 percent of the electronic equipment in the data centerfacility, and wherein the data center facility is rated at more than 10MW.

In a third aspect combinable with any of the previous aspects, thepower/cooling units include an electric generator unit generating powerfrom the working fluid and supplying waste from the working fluid to acooling unit that supplies cooling fluid to cool the electronicequipment.

In a fourth aspect combinable with any of the previous aspects, each ofthe plurality of power/cooling units includes an electric generator unitarranged to receive a working fluid and generate electricity using theworking fluid; a conduit connected to receive waste from the electricgenerator unit that is generated from the working fluid; and a coolingunit connected to receive the waste from the conduit and to providingcooling for one or more computers.

In a fifth aspect combinable with any of the previous aspects, theelectric generator unit includes a steam turbine and the waste includeshot water produced form the steam.

In a sixth aspect combinable with any of the previous aspects, thecooling unit includes an absorption chiller.

In a seventh aspect combinable with any of the previous aspects, theelectric generator unit and the cooling unit are located in a unitaryhousing with each other.

In an eighth aspect combinable with any of the previous aspects, theunitary housing is include in a rack arranged to be positioned in a rowof computer racks.

In a ninth aspect combinable with any of the previous aspects, thecooling unit includes an input positioned to obtain hot air fromcomputer racks and to supply cooled air to the same computer racks.

In a tenth aspect combinable with any of the previous aspects, a hot airinput for the cooling unit is located on a first side of the housing,and a cool air output for the cooling unit is located on a second sideof the housing that is opposite the first side of the housing.

In an eleventh aspect combinable with any of the previous aspects, theunitary housing is located above a row of racks that includes the one ormore racks of computers.

A twelfth aspect combinable with any of the previous aspects furtherincludes an exhaust connecting the unitary housing to an area externalto the data center.

In a thirteenth aspect combinable with any of the previous aspects, theexhaust includes a conduit connected to a hot air plenum in an atticabove, or an under-floor space below, racks of computers in a datacenter.

The details of one or more embodiments are set forth in the accompanyingdrawings and the description below. Other features, objects, andadvantages will be apparent from the description and drawings, and fromthe claims.

DESCRIPTION OF DRAWINGS

FIGS. 1A-1C are elevation views of example power and cooling units thatmay be used in a data center environment.

FIG. 1D is a perspective view of paired rows of computer racks havingpower and cooling units.

FIG. 1E is an elevation view of an example power and coolinginstallation.

FIG. 2 is a schematic diagram of a power-cooling unit.

FIGS. 3A and 3B show plan views of data centers using distributedpower-cooling units.

FIG. 3C shows an example of a large-domain distributed power and coolinginstallation.

FIG. 4 is a flow chart of a process for providing distributed power andcooling in a data center.

FIG. 5 is a schematic diagram of a computer system that can be used forthe operations described in association with any of thecomputer-implement methods described herein.

DETAILED DESCRIPTION

This document generally describes devices, and systems of multipledevices, that can provide distributed power generation and cooling in adata center. Such devices may include both an electrical generatorsub-system and a cooling sub-system, where the two sub-systems obtainpower from a single working fluid. For example, the working fluid may besteam and the electric generator sub-system may include a steam turbineconnected to a generator. The steam may turn partially or wholly intohot water in passing through the turbine, and such hot waste water canbe passed to an absorption chiller that can be used to provide coolingto the same computer. The two sub-systems may be joined in a singlecommon package within a common single housing, and may be procuredtogether as a single enclosed unit having a fluid input and electricaland ventilated cool-air outputs.

The inputs of the devices, the outputs of the device, or both, may betied together into a large common energy plane, where the energy source(e.g., steam or electricity) can flow readily and naturally across theentire domain without interference from components placed in the domain(such as switches, valves, transformers, etc.). For example, the workingfluid, electrical power, or both, may occupy a domain that is more thana majority, more than 75%, or more than 90% of the coverage for anentire data center facility, where such facility may have IT loads of10, 20, 30, 40, 50, 75, 90, 100, or 200 MW, or any range between any twoof these numbers (e.g., a range of 75 to 100 MW). Also, the workingfluid may occupy a similar large domain, while the electric powercreated from the working fluid in that domain may be confined to arelative large number of smaller domains. For example, the ratio ofelectric power domains to working fluid domains may be 5:1, 10:1, 20:1,50:1, 100:1, 250:1, 500:1, or 1000:1, or any range between any of thosetwo ratios such as between 10:1 and 100:1.

As one example, a data center facility may have a working fluid domainlarger than 25 or 50 MW, and the average electric domain served by thatworking fluid domain for generating the electricity being smaller than 1MW, smaller than 0.5 MW, and smaller than 0.25 MW. As noted above, suchrelatively small electric domains may reduce the size of failure domainsthere is a problem with electric power generation and/or distribution,so that a relatively large portion of a facility remains operational ifthere is an electric power failure.

FIGS. 1A-C are elevation views of example power and cooling units thatmay be used in various systems in a data center environment. FIG. 1Ashows, in particular, power/cooling units that are mounted above a rowof racks 100. As shown in the figure, the racks, exemplified by rack 116and positioned side-by-side in the row 100, hold a plurality of traysthat are each slid into open fronts of the respective racks and thateach support one or more motherboards to which are mounted or otherwiseconnected microprocessors and other support chips and other componentsfor operating as computer servers in a typical data center arrangement.An open-air space may be left between each adjacent tray in a verticalstack of trays so as to permit air to circulate around heat-generatingcomponents in each tray, so as to create convective cooling.Alternatively, or in addition, one or more portions of the componentssupported by each of the motherboards may be in contact with a thermallyconductive solid material, such as an aluminum block, and such a solidmaterial may be liquid cooled, so as to provide conductive heat removalfrom the components.

The racks may take other configurations and hold other components inappropriate implementations. For example, networking equipment, powersupplies, and other components needed to support computers in thevarious racks may also be mounted in the same racks as the computers, orin other ones of the racks, such as in every N. racks in a row. Also,although the computers are shown in this example as being horizontallypositioned on trays that are slid into the open fronts of the racks likelunch trays in cafeteria racks, the motherboards and computers may takeother forms, such as blade servers and similar configurations.Electrical and data connections may be made to the computers on thefront side, such as by way of data jacks provided on each motherboard,or on the back side, such as in the form of tight-fit blade mountingconnections into which the motherboards are each slid.

An area behind the row of racks may be used as a warm air capture plenumfor a data center. For example, cool air may enter each of the racks ona front side that is shown in the figure, may pass over components inthe rack, and may exit the rack at a back side that is opposite to thefront side. The area behind each of the racks may be enclosed so as toform such a warm air plenum that receives the air from each of the racksof computers. In certain implementations, and as shown in more detailbelow, the warm air plenum may be positioned between two rows of racksthat are parallel to each other, and may therefore receive warm airsimultaneously from both rows of racks.

Each rack may be made up of one or more vertical stacks of computers orother components. A rack in such an example is considered to be anindividually movable component, such as one mounted on caster wheels,that may be slid into place in a row of racks or may be removed, such aswhen replacing or performing maintenance on computers in the rack. Insome examples, each rack may include three stacks, and each stack mayinclude 10, 20, 30, 40, or more trays, or a number of trays in a rangebetween any two of those values. The racks may be of a height thatmaximizes density within a data center facility, while being convenientto move, deploy, and service. For example, the racks may each be aheight of about 5 feet, 7 feet, 8 feet, 9 feet, or 10 feet tall, or aheight that is between any two of such heights.

Each rack in this example, and each stack also, may receive electricalpower service and cooling service from a dedicated and localpower/cooling unit exemplified by power/cooling unit 110. In thisexample, the power/cooling unit 110 is mounted above its correspondingrack 116, such as by being physically attached to the top of rack 116and thus may be movable with rack 116, or by being mounted to thestructure of the data center and then connected to appropriatecomponents of rack 116 when rack 116 has been moved into position underthe power/cooling unit 110.

The power/cooling unit 110 is made up of a single housing, which may beapproximately the size of a hotel refrigerator, and may be prepackagedand installed by simply hooking up input and output connections for thepower/cooling unit 110. Within the housing for the power/cooling unit110, may be two zones, which may be separate or may overlap each otherin some manners. A first zone is an electrical generation section 106,in which is located a mechanism for converting a powering fluid carriedin conduit 102 into motion and a generator for producing electricityfrom that motion. For example, a steam turbine or other form of turbinemay be included in the electrical generation section 106 and may be usedto power a generator.

A cooling section 108 of the power/cooling unit 110 may include a systemfor cooling one or more fluids (e.g., circulating air) that are heatedby the components in the racks such as rack 116. The cooling section 108may, for example, cool air and/or water that is used to in turn cool thecomponents in the rack 116. For example, the cooling section 108 mayinclude an absorption chiller that may create cooling from residualsteam or hot water that is a waste product or byproduct of theelectrical generation section 106. For example, a conduit appropriatelysized may connect a steam turbine in the unit 110 to an absorptionchiller in the unit 110 so that waste heated fluid can be carried to thecooling section 108 to be used in providing cooling. Appropriate drainsmay also be included for removing water that is waste that may begenerated by the absorption chiller.

A conduit 114 is shown leading downward from the cooling section 108,and is exemplary of similar structures that may be positioned at eachrack or bay. The conduit 114 may take the form of a pipe where thecooling section 108 delivers cooling water, or a duct where the coolingsection 108 delivers cooled air. For example, where conduction coolingis being used in the rack 116, the conduit 114 may have cooling watercirculated through it and may be connected to heat sink blocks that arein contact with each computer in the rack 116. Although not shown, aseparate conduit may be used to carry warm water out of each heat sinkblock and back up to cooling section 108. Where conduit 114 is a duct,air may be distributed through apertures located at each tray in therack 116, and additional dispersal of cool air may be achieved byattaching dispersal wands to the conduit 114, where each wand may berotated into position in front of the rack 116 and a corresponding trayin the rack 116.

Each such wand may thus be in a horizontal position when in use, andapertures spread across the length of each wand may distribute cool airin the space in front of the corresponding tray. The air may then bedrawn into the rack and may pass over the respective tray so as to coolelectrical components that are supported by that tray. When maintenanceis needed on a particular tray, the corresponding wand may be rotatedupward or downward into a vertical position so as to be removed frominterfering with removal of a tray or motherboard supported by a tray.

In this manner, localized cooling may be provided to each tray in therack and in certain implementations, control may be provided at eachwand (e.g., via a motorized damper) so as to permit more granularcontrol over where cooled air is provided in a system. For example, acontrol system may be made aware of a level of computer processing thatis being performed or is to be performed by a particular server orservers in a tray, and a control valve may be adjusted on a damper for acorresponding wand so as to provide a level of airflow commensurate withthe level of computing that will be performed, and by extension with alevel of heat that will be generated in the particular tray.

Referring now to FIG. 1B, there is showing a single packagedpower/cooling unit 124 as part of a system 120 that generates heat andrequires electricity and cooling. In this example, the unit may be likeunit 110 in FIG. 1A, though here, the unit serves an entire 3-by rackrather than simply a single bay. In certain implementations, the unit124 may serve the rack 122 that is shown and a corresponding rack thatbacks up to rack 122 (and those view is blocked in this representationby rack 122).

The unit 124 is served with a working fluid, such as steam or hydrogenor natural gas, by conduit 126. Such working fluid may initially beemployed to generate electricity which may be provided to computers inrack 122 by way of electrical conductors (not shown). For example, apower strip may be provided vertically at each location where two baysof a rack meet each other, and power cords may be plugged into the powerstrip for each motherboard in a rack. The power strip may in turn have aconductor or conductors that run from it to one or more outlets in theunit 124.

A supply duct 130 is showing extending downward from the unit 124 to awarm air aisle behind rack 122. Such a warm air area may be enclosed,and may be located behind rack 122 and behind a corresponding rack thatis behind rack 122. As such, cool air may enter through an open front ofrack 122, flow across motherboards and electronic heat-generatingcomponents in rack 122, and pass out the back of rack 122 and into thewarm air space. The duct 130 may then bring the warm air up to unit 124where it may be cooled by a cooling system that is part of unit 124 andthat runs off of the remains of the working fluid that is introducedinto unit 124 (after that working fluid is first used to generateelectricity).

The cooled air may be released by way of diffuser 128, which faces inthe same direction as the front of the bays in the rack 122. The cooledair may then be drawn into the front portions of each part of the rack122 and circulated across the electronic equipment to be heated again.Such circulation may occur continuously, with the electronic equipmentin the rack 122 heating the air, and the unit 214 cooling the air. Inother implementations, air from the warm air aisle may be exhaustedoutside of a facility, and the unit 124 may draw air directly from theoutside (e.g., via a duct), or from the main occupied area of the datacenter, where makeup air may be provided from outdoors to replace anywarm air that is exhausted outdoors. Additional fans and othercomponents may be located in appropriate areas to encourage suchairflow, such as small fans at the back of each tray in the rack 122,and fans having low delta pressure at louvers that allow air into thedata center, such as large propeller fans and similar forms of fans.

Referring now to FIG. 1C, a system 140 is shown in which a combinedpower/cooling unit 146 is located in line with a row of racks ratherthan above the role of racks. In general, such an implementation maytake up space that would otherwise be occupied by computers and otherelectronic equipment, but may also be easier to access and maintain, andmay provide for a larger form factor of unit 146, and thus potentiallyadditional cooling from unit 146.

In the system 140, a rack 142 is shown that has two bays, and isadjacent the unit 146. Another rack having two bays is shown on anopposed side of unit 146. As in the other figures, the racks have openfronts so that air from a workspace of a data center may be readilydrawn into areas around electronic componentry in the racks and coolsuch componentry with a minimum air pressure drop.

The unit 146 includes an air intake 148 and air output 150, wheregenerally, air would exit through the air output 150 at a substantiallylower temperature than it entered via the air intake 148. The intake 148and output 150 may be on a same side of the unit 146 or on differentsides, such as the output 150 being on a side that faces ahuman-occupied cold aisle and the intake 148 on an opposite side fromthe output and receiving air from a hot aisle that is located behind therack 142. As with the rack in FIG. 1B, the rack 142 and other racks canbe located in a longer row of racks (e.g., like that shown in FIG. 1A)that may back up to the backs of a parallel row of racks, though with ahot aisle in between the parallel rows. In certain embodiments, chimneysor other ducts may extend upward from the unit 146 and/or the hot aisle,and may terminate above a roof of the data center or may terminate in ahot attic that collects hot air from multiple aisles and that may inturn be vented to the outdoors so that hot air may be immediatelyremoved from a data center and not be circulated. In such a situation,make-up air may be introduced into the data center facility, such as vialouvers along one wall of the facility, perhaps aided by fans, andprovided with humidification as appropriate upon the air entering thefacility.

FIG. 1D is a perspective view of paired rows of computer racks havingpower and cooling units. In the figure, a system 160 is shown in partfor a data center, where a pair of rows of racks is shown but should beunderstood to be one of multiple similar paired rows across the width ofa data center. The arrangement in this example is similar to that shownin FIG. 1C, though shown in perspective to better illustrate theexemplary arrangement of components in the data center.

The system 160 includes a pair of rows of computer racks, where eachrack is made up of two bays. A first row 164 backs up to a hot air aisle162, which is enclosed and sealed so that air in the hot air aisle doesnot mix with air in the occupied space above and in front of the row 164and other rows. Centered between the two racks shown in row 164 is acombined power/cooling unit 166, which occupies an entire bay in the row164. Each of the racks and the unit 166 may be mounted on casters orwheels that may be retracted or locked once they are moved in positionin the row 164. The unit 166 is provided with a pair of linear diffusers168 which may supply cooled air into an area of a human-occupied aislein the data center, for circulation through the human-occupied area andinto the computers mounted in the racks. The diffusers 168 may extendalmost a full height of the racks so as to provide cooled air evenly atall vertical levels so that no motherboards in the racks are starved ofcool air.

The unit 166 houses both an electric generation sub-system which maytake a form similar those described above and below, and a coolingsub-system, which may also take a form like those described above andbelow. In this example, the diffusers 168 are shown as terminating shortof the top of the unit 166 because, for example, the unit 166 may haveelectrical generating equipment in its top portion that blocks theextension of the diffusers 168 all the way to the top of the unit 166.

The unit 166 is served by a take off 172 in the form of a scheme conduitthat is attached to a main 170. Main 170 may in turn be attached andreceive steam from a larger header that may extend laterally across theend of a data center and across multiple different pairs of rows ofracks, where a main light main 170 extends parallel with each of theracks, such as above a human occupied or above 162. The steam or otherconduits may be insulated or otherwise protected to prevent damage andto prevent condensation or leaks that may otherwise drip onto sensitiveelectronic equipment in the racks 164.

FIG. 1E is an elevation view of an example power and coolinginstallation 180. In general, the installation 180 is an example inwhich a combined power/cooling unit 184 may be installed over ahuman-occupied aisle 182 in a data center facility. Although oneexemplary unit 184 is shown, multiple such units would be distributedrelatively evenly across the entire facility in a typicalimplementation, with typically multiple units in each aisle of thefacility. In a typical implementation, the units may be spaced so as tominimize the amount of ductwork or other cool-air distributionmechanisms that will be needed, such that each unit is relativelyproximate to computer racks and other equipment that it serves cool airto.

As shown in the figure, a technician is located in the cool air aisle182 performing maintenance on computers in a rack 196. As with the otherexamples discussed above, the rack 196 may have an open front, withindividual motherboards lying horizontally, and spaced apart from eachother vertically, so that air may enter through the front of the rack196, pass through the rack between the motherboards, and pass into awarm air plenum 194. The plenum 194 is positioned between the backs ofadjacent rows of computer racks and receives warmed air from suchadjacent roads.

A fan 192 is located at a top surface of the warm air plenum 194, whichmay be sealed from the cool air aisle 182 and other areas of theworkspace in the facility. The fan 192 may draw air from the warm airplenum 194 and push the air three flexible duct 190 into an attic space198. Each such warm air plenum 194 may have multiple such fans 192 alongits length, with the number of fans been selected to match a volume ofairflow needed through the racks served by the respective fans and intothe attic 198. In other embodiments, the warm air plenum 194 may beconnected to the attic space 198 along its entire length, rather thanwith relatively small individual ducts like duct 190. In additionalimplementations, air from the warm air plenum 194 may be routed downwardinto an under-floor space. In yet other implementations, attic space 198may be omitted, and the air may be exhausted directly out of the datacenter facility to the outdoors.

Different levels of air may be recirculated exhausted and or brought infresh, to replace air that is removed from the warm air plenum 194. Forexample, cooling coils may be provided to receive air from the atticspace 198, cool the air, and return the air to the workspace. Such coilsmay be part of unit 184, where a duct (not shown) may lead upward fromunit 184 into attic space 198 to draw such air for recirculation. Inother implementations, some or all of the air in attic space 198 may beexhausted to the outdoors, and unit 184 may obtain some or all of itssupply air from the workspace (and the workspace may obtain outdoorair).

The unit 184 is shown supplying electrical power to rack 196 and otherracks by way of conductors 188. In one implementation, the unit 184 mayhave electrical outlets mounted to it outer surface so that whips orcords 188 may be connected to it and connected down to power strips thatrun along the edge of rack 196. Individual conductors may then lead fromthe power strip or strips to particular ones of the computers in therack. The power supplied by unit 184 may be alternating current power,direct current power or a combination of the two, where some outputsprovide alternating current power and other outlets provide directcurrent power.

A conductor 199 is shown attached to an additional outlet from unit 184.The conductor 199 may in turn be connected in like manner to other suchcombined power/cooling unit in a facility. Current may be allowed toflow in the conductor and to be selectively provided or used by ones ofthe units that currently have excess capacity or need excess capacityfor electrical power. For example, at a particular moment in time,computers in rack 194 and any other racks served by unit 184 may requiremore electric power than unit 184 is capable of providing. In such asituation, unit 184 may be controlled to supplement the power itprovides to its associated racks with power drawn from conductor 199. Inother situations, unit 184 may provide power to conductor 199 which maybe used by other units within the facility.

Unit 184 is served with a working fluid conduit 186. Such conduit 186may carry, for example, steam or natural gas or another source of energythat may be converted into electricity and cooling by unit 184. As withconductor 199, conduit 186 may connect across multiple units, and mayconnect to a header that serves multiple rows of units. In this manner,just as conductor 199 a part of a large power plane from unit 184 toother units to which the conductor 199 is connected, the conduit 186 maylikewise be in a large plane for the provision of the working fluid. Assuch, a network of conduits may supply the working fluid flexibly inresponse to immediate demands for such fluid, with the fluid flowingreadily and naturally across the common plane of conduits. As notedabove, such use of common planes may increase the utilization level fora facility, and decrease the capital costs and sizes of distributionsystems inside the facility. In addition, where cooling units aredistributed across a facility, the facility may be implemented without aneed for duct work and other cool-air distribution components that wouldbe needed with a central air handling unit or similar system. An examplesystem showing large working fluid to electrical domains is illustratedin more detail below with respect to FIG. 3C.

While the use of working fluid and electrical interconnections acrossunits in a system via large energy planes has been discussed mostparticularly with respect to the system shown in FIG. 1E, suchinterconnection may equally apply to the systems shown in FIGS. 1A to 1Din similar manner.

FIG. 2 is a schematic diagram of a power-cooling unit 200. In general,the figure shows, broadly, subsystems that may be part of unit 200, andthe inputs and outputs of each such sub system.

Within a units 200, there may be located a power generation unit 202 andan absorption chiller tool for. The two may be mounted in a commonunitary housing, and may be manufactured in a conventional manner andshipped to a site for installation at the site where a data center islocated. In other implementations, each such subunit may be mounted inits own housing, and the two subunits may be connected directly to eachother or connected near each other by way of a conduit and otherconnecting mechanisms.

The power generation unit 204 receives an import a working fluid, suchas steam, natural gas, or hydrogen. The power generation unit 202 mayinclude a mechanism for converting the working fluid into motion, suchas rotational motion to drive a generator, fuel cell, or other generatorof electricity. One output of power generation unit 202 is electricpower, which may then be provided to electrical loads 206, such asservers, networking equipment, and other electrical loads within a datacenter facility. The electric power may take the form of old voltage ACor DC power, such as power a level of 460 V or lower. In someimplementations, the power may be generated at a first voltage andstepped down to one or more other voltages before being provided to theelectrical loads 206. For example, our may be generated a level inexcess of 100 V initially, and maybe stepped down to a level below 12 Vfor supply to the electrical loads 206. Such stepped down in voltage mayoccur inside the unit that houses the power generation unit 202, in oneor more power supply is located between the electrical loads 206 and thepower generation unit 202, or a combination of the two, such as aninitial step down in voltage of occurring in the power generation unit202 and additional step downs occurring in separate power supplies.

A second output of the power generation unit 202 is waste from theelectric generation process, such as hot exhaust steam of a lowerpressure than the steam that entered the power generation 202 and alower temperature. Alternatively, or in addition, to the provision ofsteam, hot water may also be passed from power generation unit 202 toabsorption chiller 204. The Georgian chiller 204 uses the hot enteringfluid as part of a cooling cycle to create cooling for hot water that isprovided to the absorption chiller 204 from the electrical loads 206.For example, a cooling coil may be provided that receives chilled wateror cool water from absorption chiller 204 and returns heated water toabsorption chiller 204, while receiving heated air from an area aroundthe electrical loads 206, and providing cooled air to the electricalloads 206. The particular spacing and positioning of the powergeneration unit 202, the absorption chiller 204, and the electricalloads 206 to each other may take a form similar to that shown in thefigures discussed above, or other particular forms depending on theneeds of a particular implementation.

The units 202 and 204 may be controlled by a local controller and acentral control system. Such control may be mediated by signals receivedfrom the electrical loads 206. For example, computers in a computer rackmay determine that amount of processing that they are performing andwill perform in the near future and may provide a signal indicating alevel of cooling load that will accompany such processing. Such signaledmay then be used by the unit 200 in order to modulate the operation ofthe power generation unit 202 and the absorption chiller 204.Advantageously, increases in a need for an electric power should alsoincrease the amount of fluid that is consumed by the electric generationunit 200 2N plus supplied to the absorption chiller 204, so thatadditional capacity in cooling will generally match additional needs forcooling. In other implementations, temperature sensors may alternativelyor additionally be provided in an area around the electrical loads, andreading from such sensors may be used to modulate the operation of theunit 200.

The various control mechanisms just discussed may communicate to andfrom a central control system along with similar mechanisms for otherportions of a computer data center. The central control system mayoperate to maximize utilization of computing and cooling resources in adata center facility, and allocate processing loads and bus coolingloads across such a facility. For example, the central control systemmay ensure that computing loads do not exceed maximum levels forelectrical use or for cooling capacity of different units within a datacenter. The central control system may particularly even out the amountof computing is performed by particular rack in a data center, so thatspot Dean does not occur and exceed acceptable limits on temperatures inthe computer data center.

FIGS. 3A and 3B show plan views of data centers using distributedpower-cooling units. In general, the figures show implementationssimilar to those in FIGS. 1A-D, though “zoomed out” to show morecomplete versions of rows of racks and to show a plan view so as tobetter indicate the positional relationship between the components insuch a system.

Referring now to FIG. 3A, there is shown a pair of rows of racks,including first pair 302. Pair 302 includes two parallel rows of racksthat each back up to an enclosed warm air aisle 306. As shown by arrowsin the figure, air may enter racks in each such row, such as rack 304,from a front face of the racks, and may exit through an opposed backface of the racks into the warm air aisle 306. Located at periodicpoints in each of the rows are combined power/cooling units such as unit308. Each such unit is served by a fluid-carrying conduit 312, such as atake-off from a steam pipe system, and steam main 310. The fluid fromconduit 312 may be used to power an electrical generation subunit inunit 308, and a cooling subunit in unit 308. Arrows show air exitingunit 308 into a human-occupied aisle in the data center, and such cooledair may then be drawn back into the various racks including rack 304.Air for unit 308 may be obtained from the human-occupied aisle or fromthe warm air aisle 306. In some implementations, air from the warm airaisle 306 may be exhausted outdoors rather than being recirculated, suchas by being routed upward from the aisle and into a warm air attic whichmay then be exhausted through various forms of roof vents.

Thus, in this manner, steam main 310 and associated piping may replaceall or substantially all need for cool-air distribution componentry,such as duct work, and electrical distribution componentry, such as busducts and other similar items, in the data center. Although certain airand electrical distribution may still be needed, the use of distributedlocal cooling units and power generation units may substantially reducethe need for expensive, and potentially hard-to-maintain, electrical andair distribution systems.

Referring now to FIG. 3B, there is shown a plan view of a system 320that provides cooling units above rows of racks rather than in the samelevel as the rows of racks as shown in FIG. 3A. In FIG. 3B, there isshown two pairs 322 of rows of racks similar to those shown in FIG. 3A.A human-occupied aisle 316 is provided between the fronts of theadjacent rows of racks, and a warm air aisle is provided between thebacks of the next-adjacent rows of racks, which warm air aisle is sealedfrom the human-occupied area of cooler air. Two technicians are shownstanding in two of the human-occupied aisles to better represent thelayout of such system.

A steam main 330 or other conduit for delivering a powering fluid isshown mounted above the human-occupied aisle 316. A takeoff 332 from thesteam main 330 extends over the bottom pair of rows of racks and servescombined power/cooling units that are mounted above each bay in theracks. Only one such takeoff 332 is shown here for clarity, though itwould be understood that each of the other bays would be served in asimilar manner, and the structures shown here would be repeated for eachsuch other bay.

Referring now more specifically to units at each particular bay,combined power/cooling unit 318 is positioned over a corresponding bayand includes a duct 322 that angles downward into warm air aisle 326,and a supply duct 320 that angles to supply cooled air into an area infront of the corresponding b. Other similar units may be arranged in asimilar manner. In this example, each bay is assigned its owncorresponding unit. In other implementations, each unit may supplymultiple bays.

In some implementations, a combined power/cooling unit may serve racksin the same row in addition to racks in multiple rows, including racksin a particular pair of rows that are joined back to back by a warm airaisle, or racks that face each other across a human-occupied aisle. Forexample, multiple combined power/cooling units may be situated above thehuman-occupied aisle 316 for easy access for purposes of maintenance andother use, and may blow cool air downward into the human-occupied aisle316 and obtain outside air from ductwork that extends upward toroof-mounted air intakes or other supplies of outdoor air. Such unitsmay then have outlets for receiving electrical cords that may extend toeach side of the human-occupied aisle to power strips that are mountedon the fronts of particular racks in the human-occupied aisle 316.Multiples of such units may then be spaced along the human occupiedaisle 316 as needed, where cooling and electric capacity of each unit ismatched to corresponding cooling and electrical usage density of theelectronic components in the data center along the aisle.

FIG. 3C shows an example of a large-domain distributed power and coolinginstallation 340. The installation here shows a plan view of particularcomponents in a datacenter facility 342, which is served by adistributed grid of combined power/cooling units like unit 348.

In this example, rows of racks 350 are shown in dashed lines in thefigure, and may be similar to the racks shown in FIGS. 1A-1E. In thisexample, the racks sit on slab, though they may also be mounted on anelevated floor with open space beneath or concrete with space beneath(and where mechanical components may be routed under such floor ratherthan above the racks).

Mounted higher than the racks (or below the floor) are a grid ofcombined power/cooling units like unit 348, which may be similar tothose units discussed above, such as by including a sub-unit forgenerating electric power from a fluid such as steam or hydrogen, and asub-unit for generating cooling from the waste products of the firstsub-unit, such as an absorption chiller. One row of such units is shownhere over an aisle between rows of racks, so that technicians may easilyaccess the units via lift or ladder, and so that the units to not dripon or otherwise interfere with the computers and other electronicequipment in the racks. An equal arrangement of units may be locatedabove each of the other aisles also, so that the units form an X-by-Ygrid of units.

A fluid conduit main 344 is shown entering the facility 342 at onecorner and running as a header across the open workspace area at the endof the various aisles. Taps such as conduit 346 take off from the main344 at each aisle between rows of racks and extend along such aisles andabove the racks (though the conduits 346 could also be below the floor).At each of the units, another tap may be taken (e.g., via flexibleconnector) to unit so as to serve the working fluid to the respectiveunit. Although isolation valves may be provided in the conduits, e.g.,to allow cutting off a section of the data center facility duringmaintenance, the conduits may generally be left freely open to eachother during normal operation, so that working fluid may flow freelyacross the entire facility or a substantial portion of the facility inresponse to units demanding such fluid in a particular area of thefacility 342. The particular arrangements of each unit and connectionsfrom a working fluid supply to a unit may take a variety of forms, suchas each of those shown above in FIGS. 1A-1E.

The unit 348 is connected to an electric bus 352, which in turn may beconnected (via electrical connector 354) to other electric buses in thefacility. Again, although switches and other components may be providedfor isolating portions of the electric service from each other, innormal operation, a single open plane of power may be provided (as wasthe case with the working fluid) so that power can flow freely acrossthe plane to where it is needed. In this example, the bus ducts areshown running perpendicular to the conduits and aisles, though they mayalso be routed in parallel with such other components or in otherpositional arrangements. In addition, the bust ducts may be connected tothe individual units so as to receive power from or provide power toindividual units, as needed by such units. Appropriate components may beprovided in each unit to control such provision or use of electricpower, and a central control system may provide coordinated control forthe use of such power. For example, particular racks may be allocatedamounts of processing they may perform in a given future time period,based on an anticipated amount of electric power needed to perform theoperations and a level of power within the plane allocated to a unitthat corresponds to a particular rack.

As noted above, such management of electrical power in a data center canprovide more flexibly by additionally, or alternatively, providing thedata center with a very large power plane for a data center, or a smallnumber (e.g., 1-10) of relatively large planes (e.g., 5 MW, 10 MW, 20MW, 25 MW, or 50 MW, or any number between any pair of these levels,e.g., between 10 and 25 MW). In particular, transformers and otherequipment are frequently employed in a data center and block the freeflow of electrical power from one area of the data center to another asit is needed, thus creating multiple separate power planes. For example,if a MV-to-LV transformer is placed at the end of each row of racks andone row needs excess power while another has a substandard power need,the power cannot easily pass from one row to the next because its flowwill be blocked by the transformers. If instead, a common voltage levelof power is distributed broadly in a data center in a single commonpower domain, and is stepped down to low voltage power very close to theservers (e.g., at the top of each rack or at each server), the power canmove freely throughout the medium voltage domain. Also, DC power may beused more readily when the power generation is distributed within thefacility, as the conductors needed to carry the power are relativelyshort in such an implementation. As discussed here, for example, singledomains of about 10, 20, 30, 50 75, and 100 MW in size, and in rangeswhose endpoints are identified by any combination of two of thesevalues, may be employed. Similarly, in a data center in excess of any ofthose listed sizes, a single common domain can serve at or more than 50,60, 75, 90, or 100% of the computing load in the data center facility.

The facility 342 may also have a much larger number of electric domainsthat are much smaller than the single electric domain shown here andmuch smaller than the large working fluid domain shown here. Forexample, a single working fluid domain may extend across and serve allof, 90% or more, 75% or more, or 50% or more of the IT load in facility342, whereas the average electric domain serving IT equipment may serverequal to or less than 5%, 2%, 1%, 0.5%, or 0.25%. In absolute terms, theworking fluid domain may be equal to or between any two of the followingdomain sizes for associated electricity that is generated from theworking fluid: 20, 30, 50, 60, 75, 80, 90, 100, 150, and 200 MW, whilethe average domain size for electric domains that are created from theworking fluid may be equal to or between any pair of 5 MW, 2 MW, 1 MW,0.5 MW, 0.25 MW, or 0.1 MW. Similarly, there can be equal to or between50 and 100 electric domains for the largest working fluid domain in adata center facility, or equal to or between 100 to 200, 100 to 250, 200to 400, 200 to 500, or 500 to 1000. In particular implementations, thelargest working fluid plane or domain may serve equal to or more than20, 30, 50, 75, 80, 90, or 100 percent of a data center, and the averageelectric power domain served by the working fluid domain may server lessthan 20, 10, 5, 2, 1, or 0.5% of the data center, where the data centeris equal to or larger than 10 MW, 20 MW, 30 MW, 50 MW, 75 MW, 100 MW,150 MW, or 200 MW.

In certain instances none of the electric power generating units in adomain will have its output connected to any other unit in the domain.In other instances, the most connected distributed power units that areconnected within a working fluid domain of the sizes discussed above maybe 2 units, 5 units, 10 units, 20 units, or 50 units.

FIG. 4 is a flow chart of a process for providing distributed power andcooling in a data center. In general, the process involves supplying aworking fluid to one or more power/cooling units that convert theworking fluid to electrical power and cooling of air in a space, andprovide such electric power and cooling to loads in the space, such ascomputer servers, networking equipment, and related equipment.

The process begins at box 400, where a working fluid is provided to acombined power/cooling unit. The unit may be located within a datacenter facility in a common area with computing loads of the data centerfacility. Multiple such units may be distributed throughout the facilityadjacent to the loads that they serve. For example, dozens or hundredsof such units may be distributed substantially evenly across a largedata center, with the density of the units within the data centergenerally matching the density of electric and cooling loads for theequipment in the data center facility (e.g., the units would be denserin a corner of the data center having servers that generate more heat orracks that space servers more closely to each other). The fluid may beprovided in various forms, such as low, medium, or high pressure steamsuitable for provision to commercially-available forms of powergeneration and cooling units.

At box 402, electric power is generated by processing the working fluid.In one example, a working fluid in the form of steam may be fed to aturbine that is powered by steam and is connected to an electricgenerator to provide rotational force to the generator. Such an electricgeneration pair may take a variety of familiar forms, and may providefor lightweight and efficient electric generation. The work that isperformed by the steam in turning the turbine may lower the pressure ofthe steam and the temperature of the steam, and make a cause some of thesteam to precipitate out as high temperature water.

At box 404 waste from the electric power generation step is passed fromthe power generation unit to the cooling unit. In the particular examplein which steam is used as a working fluid, such waste may be in the formof high temperature water, steam at a lower pressure than was initiallyprovided to the unit, or a combination of the two. Such waste may bepassed by a direct conduit connection between the two subunits or by amore indirect mechanism. For example, a system may supply steam tomultiple different electrical generation units, and may collect thewaste from such steam generation into a common conduit or header for allthe units, which header may in turn be connected to a plurality ofcooling units. Such a “common” connection may allow better diversity inthe availability of fluid for cooling, so that the fluid can be madeavailable to whatever unit currently needs the most cooling, even ifthat the corresponding electric generation unit that serves the samearea of the data center is not currently producing the most usable waste(e.g., because of a time lag between demand for electric power anddemand for cooling).

At box 406, cooling is generated in the cooling unit using the waste. Inthe example discussed here, such cooling may be obtained by way of anabsorption chilling process. The heat source for the absorption chillerin the main example here is steam or hot water produced from steam. Inother embodiments, such heat source may be generated by using naturalgas as a primary fluid for the electrical generation, such as using thegas to drive a turbine connected to a generator. The waste fluidprovided for the absorption chiller may then be in the form of heatedair (in addition to conductive heating) created by the combustion of thenatural gas.

At box 408, electric power is provided to the various computing loadsserved by the unit. Such provision may be by way of electricalconductors connected, such as by electric plugs, into the unit at oneend of the conductors, and connected to power supplies for motherboardsat the other end of such conductors. Additional components may also beincluded between the generator and the loads, such as electric filters,circuit breakers, switches, and transformers. The particular componentsthat are used in a particular implementation will vary based on theneeds of the data center equipment and the type of generator that isemployed. For example, AC-to-DC rectification may be needed atparticular locations, as may various levels of voltage transformation(e.g. one or more step-downs).

At box 410, cooling is provided around the computing load. Such coolingmay be provided with air or other gas, liquid, or both. For example, adiffuser may be provided with the cooling unit to spread cooled air intoa space that is in communication with air intakes for the variousservers in a data center. Such provision of cooled air may simplyinvolve dumping the cool air into an occupied cool area or aisle of adata center so that it can be readily drawn into open front faces ofcomputers in the data center. Provision of liquid cooling may be by

In this manner, the process described here may permit for theinstallation of multiple power units and cooling units distributedsubstantially evenly across a large area of a data center (e.g., servingmore than 50%, 50%, 70% 80%, or 90% of the data center). For example, alarge data center (10s of megawatts or more) may have dozens, hundredsor thousands of such units), and combinations of such units may generateelectricity and cooling at such distributed locations in the data centerfacility. The electric generation units may be located close enough toparticular racks that they serve so that power may be conveyedconveniently and economically by standard conductors (e.g., extensioncords rather than solid bus bars) The cooling units may be located so asto provide adequately even cooling coverage of the data center withoutthe need for substantial supply ductwork to carry the cooled air. And incertain implementations, particular electric generation units andparticular cooling units may be connected together and even share asingle unit, such as being part of a packaged unit that is manufacturedas a single unit off-site and installed as a single unit. Such anapproach may also substantially reduce the amount of electrical and HVACinfrastructure that needs to be installed in and contained in a datacenter space, with the former infrastructure being replaced by pipingdistribution for the powering fluid.

FIG. 5 is a schematic diagram of a computer system 500. The system 500can be used for the operations described in association with any of thecomputer-implement methods described previously, according to oneimplementation. For example, the system 500 may be used in providinglocal control for particular ones of or small groups of, combinedpower/cooling units described above, or in providing master control overan entire data center or multiple data centers of such units. Moreover,the system 500 may describe computing resources that may operate as theloads to be cooled by the systems and methods described above.

The system 500 is intended to include various forms of digitalcomputers, such as laptops, desktops, workstations, personal digitalassistants, servers, blade servers, mainframes, and other appropriatecomputers. The system 500 can also include mobile devices, such aspersonal digital assistants, cellular telephones, smartphones, and othersimilar computing devices. Additionally the system can include portablestorage media, such as, Universal Serial Bus (USB) flash drives. Forexample, the USB flash drives may store operating systems and otherapplications. The USB flash drives can include input/output components,such as a wireless transmitter or USB connector that may be insertedinto a USB port of another computing device.

The system 500 includes a processor 510, a memory 520, a storage device530, and an input/output device 540. Each of the components 510, 520,530, and 540 are interconnected using a system bus 550. The processor510 is capable of processing instructions for execution within thesystem 500. The processor may be designed using any of a number ofarchitectures. For example, the processor 510 may be a CISC (ComplexInstruction Set Computers) processor, a RISC (Reduced Instruction SetComputer) processor, or a MISC (Minimal Instruction Set Computer)processor.

In one implementation, the processor 510 is a single-threaded processor.In another implementation, the processor 510 is a multi-threadedprocessor. The processor 510 is capable of processing instructionsstored in the memory 520 or on the storage device 530 to displaygraphical information for a user interface on the input/output device540.

The memory 520 stores information within the system 500. In oneimplementation, the memory 520 is a computer-readable medium. In oneimplementation, the memory 520 is a volatile memory unit. In anotherimplementation, the memory 520 is a non-volatile memory unit.

The storage device 530 is capable of providing mass storage for thesystem 500. In one implementation, the storage device 530 is acomputer-readable medium. In various different implementations, thestorage device 530 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device.

The input/output device 540 provides input/output operations for thesystem 500. In one implementation, the input/output device 540 includesa keyboard and/or pointing device. In another implementation, theinput/output device 540 includes a display unit for displaying graphicaluser interfaces.

The features described can be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them. The apparatus can be implemented in a computerprogram product tangibly embodied in an information carrier, e.g., in amachine-readable storage device for execution by a programmableprocessor; and method steps can be performed by a programmable processorexecuting a program of instructions to perform functions of thedescribed implementations by operating on input data and generatingoutput. The described features can be implemented advantageously in oneor more computer programs that are executable on a programmable systemincluding at least one programmable processor coupled to receive dataand instructions from, and to transmit data and instructions to, a datastorage system, at least one input device, and at least one outputdevice. A computer program is a set of instructions that can be used,directly or indirectly, in a computer to perform a certain activity orbring about a certain result. A computer program can be written in anyform of programming language, including compiled or interpretedlanguages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment.

Suitable processors for the execution of a program of instructionsinclude, by way of example, both general and special purposemicroprocessors, and the sole processor or one of multiple processors ofany kind of computer. Generally, a processor will receive instructionsand data from a read-only memory or a random access memory or both. Theessential elements of a computer are a processor for executinginstructions and one or more memories for storing instructions and data.Generally, a computer will also include, or be operatively coupled tocommunicate with, one or more mass storage devices for storing datafiles; such devices include magnetic disks, such as internal hard disksand removable disks; magneto-optical disks; and optical disks. Storagedevices suitable for tangibly embodying computer program instructionsand data include all forms of non-volatile memory, including by way ofexample semiconductor memory devices, such as EPROM, EEPROM, and flashmemory devices; magnetic disks such as internal hard disks and removabledisks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Theprocessor and the memory can be supplemented by, or incorporated in,ASICs (application-specific integrated circuits).

To provide for interaction with a user, the features can be implementedon a computer having a display device such as a CRT (cathode ray tube)or LCD (liquid crystal display) monitor for displaying information tothe user and a keyboard and a pointing device such as a mouse or atrackball by which the user can provide input to the computer.Additionally, such activities can be implemented via touchscreenflat-panel displays and other appropriate mechanisms.

The features can be implemented in a computer system that includes aback-end component, such as a data server, or that includes a middlewarecomponent, such as an application server or an Internet server, or thatincludes a front-end component, such as a client computer having agraphical user interface or an Internet browser, or any combination ofthem. The components of the system can be connected by any form ormedium of digital data communication such as a communication network.Examples of communication networks include a local area network (“LAN”),a wide area network (“WAN”), peer-to-peer networks (having ad-hoc orstatic members), grid computing infrastructures, and the Internet.

The computer system can include clients and servers. A client and serverare generally remote from each other and typically interact through anetwork, such as the described one. The relationship of client andserver arises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made without departing fromthe spirit and scope of what is described. For example, the steps of theexemplary flow chart in FIG. 4 may be performed in other orders, somesteps may be removed, and other steps may be added. Accordingly, otherembodiments are within the scope of the following claims.

What is claimed is:
 1. A method for providing for conditioning of acomputer data center, the method comprising: supplying a working fluidfrom a common fluid plane to a plurality of power/cooling unitsdistributed across a data center facility in proximity to electronicequipment that is distributed across the data center facility, at leastone of the power/cooling units comprising an electric generator unitconfigured to generate electric power from the working fluid and outputwaste from the working fluid to a cooling unit to generate a coolingfluid from the waste to cool the electronic equipment, and at least oneof the power/cooling units further comprise a unitary housing thatencloses the electric generator unit and the cooling unit; convertingthe working fluid into electric power and cooling capacity at each ofthe plurality of power/cooling units; and supplying the electric powerto a common electric power plane serving a plurality of racks of theelectronic equipment in the data center facility and being served by aplurality of the power/cooling units in the data center facility.
 2. Themethod of claim 1, wherein the common fluid plane serves at least 10percent of the power/cooling units in the data center facility and thecommon electric power plane serves at most 5 percent of the electronicequipment in the data center facility, and wherein the data centerfacility is rated at 10 MW or more.
 3. The method of claim 1, whereinthe common fluid plane serves at least 90 percent of the power/coolingunits in the data center facility and the common electric power planeserves at most 10 percent of the electronic equipment in the data centerfacility, and wherein the data center facility is rated at more than 10MW.
 4. The method of claim 1, wherein the working fluid comprises avapor phase and the waste from the working fluid comprises a liquidphase.
 5. The method of claim 4, wherein the working fluid comprisessteam and the waste comprises hot water produced from the steam.
 6. Themethod of claim 4, wherein the electric generator comprises a steamturbine and the cooling unit comprises an absorption chiller.
 7. Themethod of claim 1, wherein the unitary housing is located at a locationwithin a row of racks that include the one or more racks of computers orat a location above a row of racks that includes the one or more racksof computers.
 8. The method of claim 1, further comprising exhaustingheat from the power/cooling units to an area external to the datacenter.
 9. A system comprising: a plurality of power/cooling unitsdistributed across a data center facility in proximity to electronicequipment that is distributed across the data center facility, at leastone of the power/cooling units comprising an electric generator unitconfigured to generate electric power from a working fluid coupled tothe power/cooling unit and output waste from the working fluid to acooling unit to generate a cooling fluid from the waste to cool theelectronic equipment, and at least one of the power/cooling unitsfurther comprises a unitary housing that encloses the electric generatorunit and the cooling unit; a common working fluid plane defined by aplurality of interconnected conduits arranged to serve a plurality ofthe power/cooling units; and a common electric energy plane defined by aplurality of interconnected electric conductors and served by aplurality of the power/cooling units.
 10. The system of claim 9, whereinthe common fluid plane serves at least 10 percent of the power/coolingunits in the data center facility and the common electric power planeserves at least 50 percent of the electronic equipment in the datacenter facility, and wherein the data center facility is rated at morethan 10 MW.
 11. The system of claim 9, wherein the common fluid planeserves at least 90 percent of the power/cooling units in the data centerfacility and the common electric power plane serves at least 90 percentof the electronic equipment in the data center facility, and wherein thedata center facility is rated at more than 10 MW.
 12. The system ofclaim 9, wherein the working fluid comprises a vapor phase and the wastefrom the working fluid comprises a liquid phase.
 13. The system of claim9, wherein each of the plurality of power/cooling units comprises: aconduit connected to receive waste from the electric generator unit thatis generated from the working fluid.
 14. The system of claim 13, whereinthe electric generator unit comprises a steam turbine and the wastecomprises hot water produced form the steam, and the cooling unitcomprises an absorption chiller.
 15. The system of claim 9, wherein theunitary housing is included in a rack arranged to be positioned in a rowof computer racks.
 16. The system of claim 9, wherein the cooling unitincludes a hot air input positioned to obtain hot air from computerracks and a cool air output to supply cooled air to the same computerracks.
 17. The system of claim 16, wherein the hot air input for thecooling unit is located on a first side of the housing, and the cool airoutput for the cooling unit is located on a second side of the housingthat is opposite the first side of the housing.
 18. The system of claim9, wherein the unitary housing is located above a row of racks thatincludes the one or more racks of computers.
 19. The system of claim 9,further comprising an exhaust connecting the unitary housing to an areaexternal to the data center.
 20. The system of claim 19, wherein theexhaust includes a conduit connected to a hot air plenum in an atticabove, or an under-floor space below, the rows of racks of computers inthe data center facility.